Recursive least squares method for training and pruning convolutional neural networks

Abstract Convolutional neural networks (CNNs) have shown good performance in many practical applications. However, their high computational and storage requirements make them difficult to deploy on resource-constrained devices. To address this issue, paper, we propose a novel iterative structured pruning algorithm for CNNs based the recursive least squares (RLS) optimization. Our combines inverse input autocorrelation matrices with weight evaluate prune unimportant channels or nodes each CNN layer performs next operation when testing loss is tuned down last unpruned level. can be used feedforward (FNNs) as well. The fast convergence speed of RLS optimization allows our FNNs multiple times small number epochs. We validate its effectiveness VGG-16 ResNet-50 CIFAR-10 CIFAR-100 three-layer FNN MNIST. Compared four popular algorithms, adaptively according learning task difficulty effectively even no reduction accuracy. In addition, original sample features layer.